Augmented Reality in Laparoscopic Surgery

A trend in abdominal surgery is the transition from open procedures to minimally invasive laparoscopic interventions, where visual feedback to surgical staff is only available through the laparoscope camera and direct palpation of organs is impossible. To successfully perform such sophisticated interventions, the provision of additional intraoperative feedback can be of great help to the surgical staff, especially in complex cases. This work introduces several new concepts for the application of augmented reality techniques to laparoscopic surgery. One main idea is to utilize multiple intraoperative imaging devices for the acquisition of up-to-date patient data. Optical and electromagnetic tracking systems are applied to determine the position and orientation of both rigid (mobile C-arm, laparoscope) and flexible (laparoscopic ultrasound) imaging devices. The acquired patient data is intrinsically registered to the tracked laparoscope in one common coordinate system, so it can be directly superimposed on the images of the laparoscope camera in real time without intraoperative registration steps. This intuitive superimposition can visually assist and direct the surgeon, as hidden anatomy such as vessels or tumors below the surface of an organ are revealed. The presented visualization aid can be used during critical phases in the surgical workflow such as port placement and intraoperative resection planning. Whereas superimposition for resection planning is based on intraoperative, implicitly registered imaging data, superimposition for port placement requires an interactive registration of preoperative imaging data to the patient. This interactive process is mostly automated by a newly introduced registration technique that results in a port placement procedure soundly integrated into the current surgical workflow. For resection planning and guidance, where navigated laparoscopic ultrasound can be used to acquire updated images of patient anatomy, a hybrid tracking approach including a method capable of estimating the reliability of electromagnetic tracking data is presented, which is able to automatically notify the surgical staff on possible tracking inaccuracies. The dissertation bases its validation on many experiments, including animal experiments, performed in close partnership with several surgeons.